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Mosi? methods of personality scale construction have clear statistical 

disadvantages. A hybrid method (Darlington and Bishop, 19^6) was foiind to 

increase scale validity more than any other, method, with large item pools. 

A simple modification of the Darlington-Bishop method (alge'^raically and 

conceptually similar to ridge regression, but independently derived) is proposed 

and examined; it is intended to increase scale validity by reducing validity 

shrinkage. In a^ Rouble cross-validation design, MMPI items were used to predict 

diagnostic category (schizophrenic vs. other) for 200 hospitalized patients. 

With one anomalous exception, the proposed tehcnique produced scales more valid 

- * 
than those from the original Darlington-Bishop technique. 
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'•^f . 1. ■ • 

A New Technique for Personality Scale Construction: 
Preliminary Findings 

c 

The construction ox'' personality and other inventoiy scales is an important 
topic whose statistical principles are not well understood. Three well-known 
techniques all have" clear disadvantages. The use of clinical, prior, or intuitive 
weights takes no advantage of any data which, may he available. Classical item 
analysis — in which a s^cale is constructed consisting of the items which 
correlate most highly with a criterion variable— make no use of any information 
about the intercorrelations among items. Multiple regression uses all of the 
^relevant information available from sampiU data, but suffers from problems 6t 



excessive sampling error and validity shrinkage. This is especially the case 
when large numbers of predictors are available relative to the sample size. 
In fact, in many inventory scale construction problems, the number of items 
£ may exceed the sample size n_. For instance, a sample size of 200 or 300 is 
considered *f airly large by most standards, but the best-known personality 
inventory— the MMPI— has 550 items. Multiple regression is of course impossible 
when the number of items exceeds the sample size, and very cumbersome or 
virtuadly impossible computationally when £ is very large, regardless df n. 

A fourth possible technique is stepwise mxiltiple regression. It too uses 
all of the available relevaiit information, while greatly reducing the amount 
of computation required and providing tests of reasonable length. However, 
Darlington and Bishop (1966) have shown empirically the predictive inadequacy 
of this technique in comparison with classical item analysis. A major weakness 
of stepwise multiple regression in this context is the over estimation of p^^^ 
ty r^^ (where t is the scale or test and c_ is the criterion) at each iteration, 
resulting from sampling error. 
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Darlington and Bishop (1966) proposed and empirically investigated a 
'hybrid' technique, and found that it^produced sccLles more valid than those 
obtainable «by other methods when large numbers of predictors are involved* 
Their technique begins with the construction of a provisional first-stage scale - 
according tb the item analysis approach, adopting the most valid individual items, 
(unit weights are used exclusively in the Darlington-Bishop technique). Length 
of the first-stage scale can be determined post hoc if cross-validation is 
available, by choosing the set of items which yields the most valid test* In 

most applications of this technique, cross-yalidation would probably not be; 

4 ' 

a-'/ailable; in these cases the researcher would select any number of items for 
the first-stage scal%, according to whatever guidelines or restrictions s/he 
"may te following.-" partial. coKcelations r^^'.^ (where represents the 
individTial items) are then computed for the entire item pool using the test- 
construction sample data. Items with the highest partial correlation values— 
and therefore the greatest potential for increasing the validity of the 
first-stage scale— are then added to -create a second-stage scale. 'This may' 
■involve the repetition of items already in the first-stage scale, or ""^eir 
effective deletion if' repeated with negative unit, weights . TheAecond-stage 
scale is then used to compute a seco^ set of partial correlations and the 
addition process is repeated. The optimal number of items to be added at each 
subsequent stage can be determined exactly jDost hoc if cross-validation is 
available. If not, Darlington and Bishop recommend the addition on each 
iteration of approximately 1/3 as many items as are included. in the existing 
provisional scale (counting repeat,s and deletions). They also found that scale 
cross-validity does not generally rise after inclusion of the third-stage items. 

This scale construction technique can be understood as a compromise between 
classical item analysis and stepwise regression, combining the advantages of 
each. By starting with what would be the final scale under classical item-analysis 
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procedures, the researcher can do no worse than'to match the effectiveness of 
that technique. In subsequent stages tliis technique ^do«s make use of all 
available r^elevant information about the items (inc^luding itein intercorrelations) 
by adding new items according to their partial correlation values* 

The problem of validity shrinkage due to overestimation of r^^ still ^ 
exists in the Darlington-Bishop technique, though lessened by the- great reduction 
in number of iterations performed. Th^ piresent study was undertaken to test 

the effectiveness of a modification of the Darlington-Bishop technique which 

^ t * ^ 

may raise ^ its effectiveness further by reducing validity shrinkage due to 

overestimation of p . . The modification consists of reducing the value of 

* Co * 

^ct ^^"^^^^^^ ^^^^ stage in the scale ^nstruction process. A positive 
constant less than 1.00 is , multiplied by r^'^ and the resulting value is inserted 
in place of r in the numerator of the partial correlation r^^.^* .This 
has the effect of increasing the importance of item validities relative to 
item intercorrelations in determining which items will be added to the existing 
provisional scale. The correction factor proposed for the present study is: 

1 - ^ — [1] 



which, when multiplied b r^^, can be shown to produce a more accurate estimate 

of p ^ than r + itself. 
'^ct ct 

^Examination -of the-4'ormula_for_-the.£.Qrr-ection factor reveal s at least. - 

three interesting properties f First, the size of the reduction of r^^ is 
negatively related to the overall predictive power of the item pool. Second, 
the size of the reduction of r^^ is negatively related to sample size n. 
Filially, the correction factor is mechanically simple to use. Only one computatio 
is- reeded for all operations on any given data set and criterion, and only 
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readily^available statistics are' involved in its computation. 

The original .Darlington-Bishop technique and classical item analysis can 

be reformulated in terms of the value each implicitly assigns to thi^s correction 

factor. Item analysis uses a value of 0.0 in^all cases, so that potential additions 

are evaluated only in terms of 'their predictive validities. The Darlington-Bishop 

technique always assxmes a value of l.OO, The modification proposed here can 

« 

thus be interpreted as an intermediate approach in relation to these other two 
techniques. It would seem plausible that other values for the correction factor 
(between o,0 and 1,0) could also increase scale validity over the levels 
^ obtained by adoption of either extreme value (i»e., item analysis or the 
Dcurlington-Bishop method), 

A certain modification of the Darlington-Bishop technique is also suggested 
"by the theory of ridge regression (Hoerl & Kennard, 1970; Price, 1977). We 
shaU give only the justification of the ridge regression approach here, since it 
is simpler. It can he shown algebraically that redge regression is equivalent 
to the foy.owing procedure: multiply all observed correlations in a data set, 
both item validities and item intercorrelations, by a constant l/(l+lc) ; 
then run the regression in the nomal way on these adjusted correlations. It cai 

be shown that the procedure described in the preceding paragraphs is algebraically 

% 

almost equivalent to that procedure. Thus the proposed modification can be 
considered^as suggested by the theory of ridge regression. With very large valuej 
of ridge regr ession becomes computationally prohibitive, whereas the "echnique 
described above does not* 

Method 

Subjects and item pool : .MMPI responses of 96 diagnosed schizophrenics and 
lOU non-schizophrenic psychiatric patients were used in the study. These data 

6 



^5. 

wer^ taken ft^om a larger sample made available to the investigators by Albert 
Rosen, vhich sample was also used in one of the studies reported by Darlington 
and Bishop. Two subsamples of 100 individuals were created, each including hQ 
diagnosed schizophrenics and 52 ' non-schizophi-enic patients. For budgetary 
reaso^s, the .first 150 of the 550 MMPI items were adopted as -the^ item pool for 
' this study. This procedure is^ likely to weaken rather than strengthen the 
effectiveness of the technique under study, as the problem of validity shrinkage 
increases with larger item poo?s. Parallel analyses were also undertaken using 
only the first 90 MI4PI items. 

Y ^SSlBSL- A double cross-validation design was adopted, yith scales constructed 
in each subsample cross-validated in the other subsample. Parallel tests were 
conducted on the complete 150-item pool and the first 90 items of the pool. 
^The original study design called for three different scale-construction techniques 
to be used on each item pool with each subsample: classical item analysis, the ^ 
Darlington-Bishop technique, and the modification of the latter obtained by 
nniltipiyin^ r^^ values by [l]. For the two subsamples the actual correction 
factor values obtained were ,k23h and .3303. A fourth 'technique' was subsequently 
'invented,' using a correction factor value of .7000 in-order to sample the 
full range of possible values more evenly and completely. 

Procedure ; Product-moment correlations were used as indices of validity.- 
Unit weights (positive and negative) were used exclusively in constructing each 
scafe. Cross-validation of all potential scales was undertaken so as to achieve 
optimum performances for each technique. Up to three iterations were performed 
on each data set for each technique, e^ccapt classical item analysis for which the 
first-stage test is certain to be optimal by that method. The scale construction 
steps for all four techniques are described above. 
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' ' Results and Discussion 

The results of each of "sixteen scale constructions (four -tTechniques applied • 
to four different data sets) are presented in'Tahle 1. In the data set comprised 
"by subsample A with a 90-ltem pool, the scale produced "by classical item analysis 
contained four Items. None of the three iterative procedures was able to increase 
the validity of that foxir-item scale. This finding is somewhat surprising., in 
that personality scsJ-es using MMPI or similar items are generally much longer. ' 
The other 12 completed scales included 'from 12 to 3U items, more in keeping with 
common experience in this area. In the other three data sets, the Dwrlington- 
Bishop technique yielded more valid scales (compared with item analysis-) twice. ^ 
The technique proposed in this paper perfonmed "better than classlceJ. item analysis, 
better than the Darlington-Bishop technique , and better than or equal to the 
iiftermediate-correction-factor-value 'technique' in all three cases. 

The percentage increases in scale validity obtained by the proposed modification 
technique are not large in this study. However, the percentage increases obtained 
• here by the use of the original Darlington-Bishop technique are consistently 
smaller (0% to 2%) than those obtained in the 1966 study to 22%) where all 
550 MMPI items' were used. Thus there Is reason to believe that the proposed 
modification also would have greater beneficial impact ;under more typical 
personality scale construction conditions. Further research is now underway to 
more thoroughly test the properties of this technique. Its simplicity in use, 
its conceptual and algebraic similarity to ridge regression, its performance in 
the present study, and the previously-established value of the original 
Darlington-Bishop technique, together suggest that this modification will prove 
to be a useful supplement to existing inventory scale construction technology. 
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> \ • ^ . . * Table. 1. | * ^ • 

Performancek of Pour Different Scale Construction Techniques Applied to Four Data Sets 
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